<html>
<head><meta charset="utf-8"><title>self_profile_alloc_query_strings slow? · t-compiler/wg-self-profile · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/index.html">t-compiler/wg-self-profile</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html">self_profile_alloc_query_strings slow?</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="199321772"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings%20slow%3F/near/199321772" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html#199321772">(May 31 2020 at 21:25)</a>:</h4>
<p>I'm taking a look at <a href="https://github.com/rust-lang/rust/issues/72837">#72837</a>, and running rustc with <code>-Z self-profile-events=default,query-keys -Zself-profile</code> results in a profile dominated by <code>self_profile_alloc_query_strings</code> taking 40% of the total time. Is that expected to be so slow?</p>



<a name="199322113"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings%20slow%3F/near/199322113" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html#199322113">(May 31 2020 at 21:37)</a>:</h4>
<p>query key collection is pretty slow, but I wouldn't have expected it to be <em>that</em> slow... <span class="user-mention" data-user-id="125250">@Wesley Wiser</span>, could it be the codegen unit thing you added recently?</p>



<a name="199322309"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings%20slow%3F/near/199322309" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html#199322309">(May 31 2020 at 21:43)</a>:</h4>
<p>I haven't actually checked the query keys, but it's possible that they are just gigantic</p>



<a name="199322454"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings%20slow%3F/near/199322454" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html#199322454">(May 31 2020 at 21:47)</a>:</h4>
<p>you can probably e.g. <code>strings</code> the generated file as a quick check</p>



<a name="199322511"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings%20slow%3F/near/199322511" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> andjo403 <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html#199322511">(May 31 2020 at 21:49)</a>:</h4>
<p>from where it was introduced in <a href="https://github.com/rust-lang/rust/pull/67397">https://github.com/rust-lang/rust/pull/67397</a> "This PR makes self-profiling able to record query keys. The implementation is not as efficient as it could be yet (all query keys except for DefIds cause string data to be duplicated) and the rendered strings could be nicer too" so think is expected to be slow</p>



<a name="199322859"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings%20slow%3F/near/199322859" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> andjo403 <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html#199322859">(May 31 2020 at 22:00)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> the codegen unit thing that was added shall only be logged if self-profile-events contains args or function-args so that shall not be part of the problem and that event is not triggered that often only once per cgu</p>



<a name="199322862"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings%20slow%3F/near/199322862" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html#199322862">(May 31 2020 at 22:00)</a>:</h4>
<p>ah okay</p>



<a name="199327117"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings%20slow%3F/near/199327117" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wesley Wiser <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html#199327117">(Jun 01 2020 at 00:10)</a>:</h4>
<p>It depends a lot on what the rustc session you're profiling looks like. For most sessions, the majority of time is spent in LLVM and there aren't (comparatively) that many query keys to be recorded. If the middle part of rustc is what's slow, we could easily be generating millions of queries and recording all of that data is slow. Usually dumping the query keys happens while LLVM is running so it doesn't really add much to the wall clock time.</p>



<a name="204233879"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings%20slow%3F/near/204233879" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html#204233879">(Jul 17 2020 at 17:22)</a>:</h4>
<p><span class="user-mention" data-user-id="125250">@Wesley Wiser</span> FWIW I intentionally only profile <code>cargo check</code> runs</p>



<a name="204233941"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings%20slow%3F/near/204233941" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html#204233941">(Jul 17 2020 at 17:23)</a>:</h4>
<p>(not sure how relevant it is to the discussion, just wanted to offer that data point)</p>



<a name="204233979"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings%20slow%3F/near/204233979" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> eddyb <a href="https://rust-lang.github.io/zulip_archive/stream/187831-t-compiler/wg-self-profile/topic/self_profile_alloc_query_strings.20slow.3F.html#204233979">(Jul 17 2020 at 17:23)</a>:</h4>
<p>(running LLVM is a waste of time in that case since what I'm looking for is ways to optimize trait system stuff)</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>